The present invention relates to a rangefinder for measuring the three-dimensional (3D) shape or obtaining range information of an object.
FIG. 24 illustrates an exemplary configuration for a prior art rangefinder. The rangefinder shown in FIG. 24 captures a set of images of an object by projecting light onto the object, thereby measuring the 3D shape of the object based on the principle of triangulation. The rangefinder shown in FIG. 24 was designed to operate in real time.
As shown in FIG. 24, the rangefinder includes laser light sources 101A and 101B, half mirror 102, light source controller 103, rotating mirror 104, rotation controller 105, lens 107, wavelength separating filters 108A and 108B, CCD""s 109A, 109B, 109C, signal processors 110A, 110B, 111, distance calculator 112 and controller 113. The wavelengths of laser beams emitted from the light sources 101A and 101B are slightly different from each other. The laser beams outgoing from the light sources 100A and 101B are combined with each other at the half mirror 102. The intensities of the beams emitted from the light sources 101A and 101B are controlled by the light source controller 103. The rotating mirror 104 is provided to scan the combined laser radiation and the rotation thereof is controlled by the rotation controller 105. The laser radiation, which has been projected onto an object 106 and reflected therefrom, is condensed by the lens 107, selectively passed through the wavelength separating filters 108A and 108B and then incident on the CCD""s 109A, 109B and 109C. The filters 108A and 108B separate parts of the reflected radiation with the same wavelengths as those of the light sources 101A and 101B, respectively. The CCD""s 109A and 109B capture monochromatic images of the object 106, while the other CCD 109C captures a color image of the object 106. The signal processors 110A, 110B and 111 are provided for these CCD""s 109A, 109B and 109C, respectively. The distance calculator 112 obtains, by computation, information about the distance or shape of the object 106 based on the intensities of the beams incident on the CCD""s 109A and 109B. And the controller 113 synchronizes or controls the overall operating timing of this rangefinder.
Hereinafter, it will be described how the rangefinder shown in FIG. 24 operates.
The laser light sources 101A and 101B emit laser beams with slightly different wavelengths. These laser beams are in the shape of a slit with an optical cross section crossing the scanning direction of the rotating mirror 104 at right angles. That is to say, if the rotating mirror 104 scans the laser beams horizontally, then the slitlike beams extend vertically.
FIG. 25 illustrates how the intensities of the laser beams projected from these two light sources 101A and 101B change with wavelength. As can be seen from FIG. 25, the beams emitted from these light sources 101A and 101B have close wavelengths. This is because the reflectance of the object 106 is less dependent on the wavelength in such a case. As described above, the laser beams emitted from the laser light sources 101A and 101B are combined into a single beam at the half mirror 102. Then, the combined beam is scanned at the rotating mirror 104 toward the object 106.
The rotation controller 105 drives the rotating mirror 104 on a field-by-field basis, thereby scanning the laser beams that have been projected from the light sources 101A and 101B. In this case, the light source controller 103 changes the intensities of the beams emitted from these light sources 101A and 101B as shown in FIG. 26(a) within the cycle time of one field. That is to say, the intensities of the laser beams are changed while the rotating mirror is driven.
The laser radiation that has been reflected from the object 106 is condensed by the lens 107 toward the CCD""s 109A, 109B and 109C. The wavelength separating filter 108A selectively transmits only a part of the laser radiation with the same wavelength as that of the beam emitted from the light source 101A but reflects the remaining parts of the radiation with other wavelengths. On the other hand, the wavelength separating filter 108B selectively transmits only a part of the laser radiation with the same wavelength as that of the beam emitted from the light source 101B but reflects the remaining parts of the radiation with other wavelengths. As a result, respective parts of the laser beams that have been projected from the light sources 101A and 101B onto the object 106 and then reflected therefrom are incident on the CCD""s 109A and 109B, respectively. And the rest of the laser radiation with other wavelengths is incident onto, and captured as a color image by, the CCD 109C.
The signal processors 110A and 1108 perform signal processing on the outputs of the CCD""s 109A and 109B as is done for an ordinary monochromatic camera. On the other hand, the signal processor 111 performs signal processing on the output of the CCD 109C as is done for an ordinary color camera.
The distance calculator 112 obtains distances for respective pixels based on the outputs of the CCD""s 109A and 109B, the baseline length and the coordinates of the pixels.
Specifically, based on the outputs of the CCD""s 109A and 20109B, the distance calculator 112 calculates an intensity ratio of the reflected parts of the laser radiation. The relationship between the intensity ratio and the scanning time is known. Accordingly, if an intensity ratio is given, then a specific point in time during one scan period can be estimated automatically. And the angel xcfx86 of rotation of the rotating mirror 104 can be derived from the specific point in time. For example, as shown in FIG. 26(b), supposing the intensity ratio is Iao/Ibo, then the scanning time is estimated to be t0, from which the angle xcfx86 of rotation of the rotating mirror 104 is known. This angle xcfx86 corresponds to the angle formed by a visual axis, which extends from the light source to the object, with the X-axis.
The relationship between the scanning time t and the angle xcfx86 of rotation of the rotating mirror 104 is already known, too. Thus, a characteristic table representing the relationship between the intensity ratio Ia/Ib and the angle xcfx86 of rotation may be prepared by substituting the angle xcfx86 of rotation for the axis of abscissas of the graph shown in FIG. 26(b). In such a case, the angle xcfx86 of rotation can be derived directly from the given intensity ratio without using the scanning time t.
FIG. 27 diagrammatically illustrates how the distance calculator 112 derives the distance. In FIG. 27, O is the center of the lens 107, P is a point of incidence on the object 106 and Q is a point at which the axis of rotation of the rotating mirror 104 is located. For the illustrative purposes, the CCD 109 is illustrated as being located closer to the object 106. The distance between the CCD 109 and the center O of the lens 107 is identified by f. Supposing the baseline length between the points O and Q is L, the angle formed by a visual axis QP with the X-axis within the XZ plane is xcfx86, the angle formed by another visual axis OP with the X-axis within the XZ plane is xcex8 and the angle formed by still another visual axis OP with the Z-axis within the YZ plane is xcfx89, the three-dimensional coordinates (X, Y, Z) of the point P are given by the following equations:
X=Lxc2x7tan xcfx86/(tan xcex8+tan xcfx86)
Y=Lxc2x7tan xcex8xc2x7tan xcfx86xc2x7tan xcfx89/(tan xcex8+tan xcfx86)
Z=Lxc2x7tan xcex8xc2x7tan xcfx86/(tan xcex8+tan xcfx86)
In these equations, the angle xcfx86 can be derived from the intensity ratio of the reflected beams that are being monitored by the CCD""s 109A and 109B as described above. On the other hand, the angles xcex8 and xcfx89 can be derived from the coordinates of pixels on the CCD""s 109A and 109B.
By calculating the three-dimensional coordinates (X, Y, Z) in accordance with these equations for all the pixels in each of the images captured by the cameras, 3D information about the object can be obtained. Also, if only the Z coordinates are derived for each image, then a range image can be obtained.
A cost-reduced rangefinder, whose light source section does not perform any mechanical operation, was also proposed. The rangefinder of this type includes no sweep means such as laser light sources or rotating mirrors. Instead, the rangefinder includes a light source section that can project multiple beams with mutually different two-dimensional (2D) radiation patterns.
Specifically, this alternative rangefinder projects multiple beams with different 2D radiation patterns onto an object on a timing-sharing basis, makes cameras capture the images of the object as represented by the reflected beams thereof and estimates the distance of the object based on the intensities of the reflected beams. The rangefinder estimates the distance by deriving three-dimensional information about the object based on the coordinates of the images and in accordance with the relationship between the intensity ratio or the intensities of the reflected beams and the angles of projection of beams irradiated.
The rangefinder of this type can perform three-dimensional measurement much more stably and reliably, because the entire measurement process is implemented by electronic operations. In addition, since this rangefinder needs no mechanical operations, the cost thereof is relatively low.
The prior art rangefinder, however, has the following drawbacks.
The rangefinder including the light source section for projecting multiple beams with mutually different 2D radiation patterns ordinarily uses xenon flash lamps as its light sources. However, the intensity of a beam emitted from such a flash lamp is variable by about xc2x15% or less every time the beam is emitted. Accordingly, even when the same object is imaged, the intensity of a beam reflected therefrom or the intensity ratio is changeable. In addition, variation is also found among individual flash lamps. Thus, the distance of the object cannot be estimated accurately without taking those variations into account.
In the rangefinder with the laser light sources on the other hand, each of the beams emitted from the light sources is vertically magnified by a lens system, transformed into a slitlike beam and then made to sweep the object using a rotating mirror. Such a rangefinder estimates the distance of the object while assuming the intensity of the projected beam is variable only horizontally but is constant vertically. Actually, though, the intensity of the slitlike beam is non-uniform due to the effects of lens shading. Specifically, since the intensity of the beam locally attenuates when passing through the lens system, the intensity at the edges of the slit is weaker than the intensity at the center of the slit. That is to say, since the intensity is also variable vertically, there is no longer one-to-one correspondence between the intensity ratio of the reflected beams monitored and the directions in which the beams were projected, thus making it difficult to estimate the distance accurately. A similar problem might be caused even when multiple beams with different 2D radiation patterns are projected.
It is therefore an object of the present invention to provide a rangefinder that can perform highly accurate 3D measurement of an object without being affected by any possible variation in intensity of projected light.
It is also an object of the invention to provide a rangefinder that can measure the 3D shape of an object very accurately even if there is no one-to-one correspondence between the optical characteristic of reflected light and the direction in which light was projected in an image captured by a camera.
Specifically, an inventive rangefinder includes: a light source for projecting light onto an object such that an intensity of the projected light varies on the object at least from place to place; a camera for capturing an image of the object by using part of the projected light, which part has been reflected from the object; and a 3D information generator for generating 3D information of the object from the reflected light image captured by the camera. The 3D information generator includes a light intensity corrector. The corrector corrects an intensity of the reflected light image according to correction information representing an intensity of the projected light, and generates the 3D information using an image that has been corrected by the light intensity corrector.
In the rangefinder of the present invention, the intensity of the reflected light image captured by the camera is corrected by the light intensity corrector according to the correction information representing the intensity of the projected light. And the 3D information generator generates the 3D information using an image that has been corrected by the light intensity corrector. Thus, even if the intensity of the projected light has changed, a corresponding variation in intensity of the reflected light image is compensated for by the light intensity corrector. As a result, the rangefinder can obtain highly accurate 3D information without being affected by any variation in intensity of the projected light.
In one embodiment of the present invention, the light source may project multiple light beams with mutually different projection patterns. The light intensity corrector may correct respective intensities of reflected light images corresponding to the projected beams. And the 3D information generator may generate the 3D information based on a ratio of intensities of the reflected light images that have been corrected by the light intensity corrector.
In another embodiment of the present invention, the rangefinder may further include a light receiver for detecting an intensity of the light that has been projected from the light source and outputting the intensity as the correction information.
In still another embodiment, the light intensity corrector may use, as the correction information, brightness in a predetermined part of the image captured by the camera.
In yet another embodiment, the light intensity corrector may perform offset processing on the reflected light image by subtracting black level components from image data and then make the correction.
Another inventive rangefinder includes: a light source for projecting light onto an object such that a characteristic of the projected light varies depending on a direction in which the light has been projected; a camera for capturing an image of the object by using part of the projected light, which part has been reflected from the object; and a 3D information generator for generating 3D information of the object from the reflected light image captured by the camera. The 3D information generator includes multiple lookup tables, each describing a correspondence between the characteristic of the reflected light and the direction in which the light has been projected, selects at least one of the lookup tables for each pixel location in the image captured by the camera and generates the 3D information by reference to the lookup table(s) selected.
According to the present invention, the 3D information generator includes multiple lookup tables, each describing a correspondence between the characteristic of the reflected light and the direction in which the light has been projected, and selects one of the tables for each pixel location in the image captured by the camera. Thus, even when one-to-one relationship between the optical characteristic of the reflected light and the direction in which the light has been projected is not met in the image captured by the camera, highly accurate 3D information can still be obtained. This is because one of the lookup tables, which has been selected for each pixel location, is consulted.
In one embodiment of the present invention, the lookup tables may be provided for at least some epipolar lines in the image captured by the camera. On each and every epipolar line, one-to-one correspondence is always met between the optical characteristic of the reflected light and the direction in which the light has been projected. Thus, it would be more efficient and cost effective if the lookup tables are provided for the respective epipolar lines. As a result, highly accurate 3D information can be generated with reasonable computational cost and storage capacity consumed.
In this particular embodiment, the 3D information generator may newly make a lookup table for pixels on one of the epipolar lines that is associated with none of the lookup tables by performing interpolation on at least two of the existent lookup tables associated with corresponding ones of the epipolar lines near the epipolar line on target.
As an alternative, the light source and the camera may be placed such that a line connecting the light source and the camera together matches an X-axis of a world coordinate system and is parallel to an x-axis of a camera coordinate system. In such a case, the epipolar lines are parallel to the x-axis in the image captured by the camera. As a result, the lookup tables can be made and distance can be estimated far more easily.